Claims 

1 . An object-tracking device for tracking an object based on image 
information, comprising: 

a characteristic-quantity synthesizing means adapted to synthesize 
characteristic quantities of objects representative of characteristic quantities of 
5 respective objects included in said image information for generating synthesized 
characteristic quantities; and 

a correspondence-establishing means for establishing correspondences 
between object zones and objects on the basis of degrees of similarity between 
characteristic quantities of said object zones and said synthesized characteristic 
1 0 quantities, wherein said object zones refer to the zones that are extracted from 
said image information and include the objects of interest. 



2. An object-tracking device according to claim 1 , wherein said 
characteristic-quantity synthesizing means is adapted to synthesize 
characteristic quantities for each of all required combinations of a plurality of 
objects to generate said synthesized characteristic quantities, and 

5 said correspondence-establishing means establishes correspondences 

between objects and object zones through comparing each of said synthesized 
characteristic quantities generated by said characteristic-quantity synthesizing 
means and zone characteristic quantities representative of the characteristic 
quantities of object zones. 

0 

3. An object-tracking device according to claim 2, provided with: 

an object-zone extracting means for extracting said object zones from 
said image information and providing the object-zone information that includes 
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the image information about said object zones, 

a state-of-tracking deciding means for deciding the states of tracking of 
individual objects or object zones, wherein said state-of-tracking means relative 
positions of each object with respect to other objects, and 

a characteristic-quantity generating means for generating said zone 
characteristic quantities and object characteristic quantities through the use of 
said image information, said object-zone information and the decision results 
effected by said state-of-tracking deciding means, 

wherein said characteristic-quantity synthesizing means generates synthesized 
characteristic quantities through the use of said object characteristic quantities 
and the decision results effected by said state-of-tracking deciding means. 

4. An object-tracking device according to claim 3, wherein said 
state-of-tracking deciding means decides the states of tracking of respective 
objects or object zones based on the object-zone information and the 
correspondence information that has been determined that indicates the 
corresponding relationship between the object zones and objects prior to the 
present to provide the first zone-correspondence information that indicates the 
corresponding relationship among the object zones and objects and said states 
of tracking, 

said characteristic-quantity generating means generates zone 
characteristic quantities and object characteristic quantities based on the 
current image information, said object-zone information, said first 
zone-correspondence information and said correspondence information that 
has been determined, 

said characteristic-quantity synthesizing means generates synthesized 
characteristic quantities that serve as candidates to be placed in the 
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corresponding relationship to individual object zones based on said object 
characteristic quantities and said first zone-correspondence information to 
provide synthesized characteristic-quantity information, wherein said 
synthesized characteristic-quantity information is the information that includes 

20 synthesized characteristic quantities and the corresponding relationship 
between the synthesized characteristic quantities and objects used for the 
generation of said synthesized characteristic quantities, and 
said correspondence-establishing means includes a 
correspondence-determining means that associates objects and object zones to 

25 place in the corresponding relationship based on said first 

zone-correspondence information, zone characteristic-quantity information that 
is the information indicative of said zone characteristic quantities and said 
synthesized characteristic-quantity information to provide said correspondence 
information that has been determined in the present time 

30 

5. An object-tracking device according to claim 3, wherein said state of 
tracking includes at least one of or a combination of: a stand-alone state in 
which only a single object resides in an object zone; a crossover state in which 
a plurality of objects correspond to a single object zone; and a state of parting 

5 that is a transient state in which a single object zone is parted into a plurality of 
object zones. 

6. An object-tracking device according to claim 4, wherein said state of 
tracking includes at least one of or a combination of: a stand-alone state in 
which only a single object resides in an object zone; a crossover state in which 
a plurality of objects correspond to a single object zone; and a state of parting 

5 that is a transient state in which a single object zone is parted into a plurality of 



object 2ones. 



7. An object-tracking device according to claim 4, wherein said 
characteristic-quantity generating means 

generates zone characteristic quantities, each including at least one of 
or one of combinations of a color histogram, area, image template and color 
5 histogram normalized with respect to said area, of the object zone, and 

finds an object zone corresponding to the object of interest from the first 
zone-correspondence information and provides at least one or one of 
combinations of a color histogram, area, image template and color histogram 
normalized with respect to said area of the object zone as an object 
1 0 characteristic quantity. 

8. An object-tracking device according to claim 4, wherein said 
state-of-tracking deciding means includes an object-zone storing means for 
storing the object-zone information, 

an object-tracking means for tracking an object based on said 
5 object-zone information, the correspondence information that has been 

determined and the object-zone information prior to the present that is provided 
from said object-zone storing means and further providing a second 
zone-correspondence information that indicates the correspondences between 
objects and object zones, and 
10 a state-deciding means for deciding the states of tracking of objects 

based on said second zone-correspondence information, said object-zone 
information and said object-zone information prior to the present and providing 
said first zone-correspondence information. 
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9. An object-tracking device according to claim 8, wherein said 
state-deciding means, based on at least one of or one of the combinations of 
the correspondences between objects and object zones, distances between 
object zones and continued periods of separation of said object zones, obtained 

5 from said second zone-correspondence information and object-zone information, 
groups the objects that have a common region in their corresponding object 
zones to sort the objects and corresponding object zones into one class, and 
sorts the object, which differs in the corresponding object zone from any other 
objects, and the object zone corresponding thereto into one class to sort the 
1 0 objects and object zones into a plurality of classes, and decides the state of 
tracking on the basis of the sorted classes. 

10. An object-tracking device according to claim 9, wherein said state of 
tracking includes the state of parting that is a transient state through which an 
object zone parts into a plurality of object zones, 

said state-deciding means decides that, if two or more object zones are 
5 included in a sorted class, then the class meets the condition of being in a state 
of parting, and that, if a class meets the condition of being in a state of parting, 
the states of tracking of the objects and object zones included in the class are 
the state of parting. 

11. An object-tracking device according to claim 1 0, wherein if the sorted 
class meets the condition of being in the state of parting and if the sorted class 
meets at least one of or one of the combinations of the conditions that two or 
more objects are included in said class, that each of the distances between the 

5 object zones included in said class exceeds a predetermined threshold and that 
continued periods of separation of the object zones included in said class 
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exceed a predetermined threshold, said state-deciding means decides that the 
states of tracking of the objects and object zones included in the class are said 
state of parting. 



12. An object-tracking device according to claim 10, wherein said state of 
tracking includes a state of parting and a stand-alone state in which a single 
object resides in an object zone, and 

if the sorted class includes only one object and if the states of tracking 
5 of the object and the object zone included in said class are not the state of 

parting, then said state-deciding means decides that the states of tracking of the 
object and the object zone included in said class are the stand-alone state. 

13. An object-tracking device according to claim 11 , wherein said state of 
tracking includes a state of parting and a stand-alone state in which a single 
object resides in an object zone, and 

if the sorted class includes only one object and if the states of tracking 
5 of the object and the object zone included in said class are not the state of 

parting, then said state-deciding means decides that the states of tracking of the 
object and the object zone included in said class are the stand-alone state. 

14. An object-tracking device according to claim 10, wherein said state of 
tracking includes a state of parting and also a crossover state in which a 
plurality of objects are in corresponding relationship to a single object zone, and 

if a sorted class includes two or more objects and if the states of 
5 tracking of the objects and the object zones included in said class are not the 
state of parting, said state-deciding means decides that the states of tracking of 
the objects and the object zones included in said class are the crossover state. 
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1 5. An object-tracking device according to claim 1 1 , wherein said state of 
tracking includes a state of parting and also a crossover state in which a 
plurality of objects are in correspondence relation to a single object zone, and 

if a sorted class includes two or more objects and if the states of 
5 tracking of the objects and the object zones included in said class are not the 
state of parting, said state-deciding means decides that the states of tracking of 
the objects and the object zones included in said class are the crossover state. 

16. An object-tracking device according to claim 4, wherein said 
characteristic-quantity generating means includes: 

a characteristic-quantity extracting means for extracting zone 
characteristic quantities from the image information, object-zone information 
5 and the first zone-correspondence information and providing the zone 

characteristic-quantity information that is the information indicative of said zone 
characteristic quantities; 

characteristic-quantity storing means for storing object characteristic 
quantities and selecting the stored object characteristic quantities to supply the 
1 0 selected object characteristic quantities, as required, and 

characteristic-quantity updating means for updating said object 
characteristic quantities stored in said characteristic-quantity storing means 
based on said zone characteristic-quantity information, said first 
zone-correspondence information or correspondence information that has been 
1 5 determined and the object characteristic quantities generated prior to the 
present. 

1 7. An object-tracking device according to claim 1 6, wherein 
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said state of tracking includes the state of parting that is a transient 
state through which an object zone parts into a plurality of object zones, and 

said characteristic-quantity extracting means includes, in zone 
5 characteristic-quantity information, the information indicating that there is no 
need for establishing correspondences to objects for the object zones that 
represent the states other than the state of parting while in their tracking states, 
and 

said correspondence-determining means excludes, from the 
1 0 establishment of the corresponding relationship, the object zones indicated in 
said zone characteristic-quantity information as there is no need to establish 
corresponding relationship to objects. 

1 8. An object-tracking device according to claim 1 6, wherein said state of 
tracking includes a stand-alone state in which a single object resides in an 
object zone, and 

said characteristic-quantity updating means 
5 decides whether or not the state of tracking of an object is the 

stand-alone state on the basis of the first zone-correspondence information or 
the correspondence information that has been determined, and 

if the state of tracking of the object is the state other than the 
stand-alone state, does not update the object characteristic quantities stored in 
1 0 said characteristic-quantity storing means. 

19. An object-tracking device according to claim 4, wherein said 
characteristic-quantity synthesizing means 

determines all possible combinations of objects and object zones based 
on the object characteristic quantities generated by said characteristic-quantity 
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5 generating means and the first zone-correspondence information, and 

synthesizes object characteristic quantities only for the determined 
combinations of objects and object zones to generate synthesized characteristic 
quantities. 

20. An object-tracking device according to claim 4, wherein 
characteristic-quantity synthesizing means calculates the synthesis ratios that 
are coefficients for adjusting the ratios at which the object characteristic 
quantities are synthesized, and generates synthesized characteristic quantities 

5 on the basis of said synthesis ratios and object characteristic quantities. 

21 . An object-tracking device according to claim 4, wherein said 
characteristic-quantity synthesizing means receives zone characteristic 
quantities as well as object characteristic quantities from the 
characteristic-quantity generating means, calculates synthesized characteristic 

5 quantities depending on desired synthesis ratios on the basis of the received 
zone characteristic-quantity information and object characteristic quantities, and 
provides the synthesized characteristic quantity for the synthesis ratio that 
yields the highest of all degrees of similarity between the calculated synthesized 
characteristic quantities and the zone characteristic quantities. 

10 

22. An object-tracking device according to claim 4, wherein said state of 
tracking includes a state of parting that is a transient state through which an 
object zone parts into a plurality of object zones, and 

said characteristic-quantity synthesizing means generates synthesized 
5 characteristic quantities only for the object zones that are indicated as having 
the state of parting as their states of tracking. 
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23. An object-tracking device according to claim 4, wherein said object 
characteristic quantity includes an area of an object, and 

said characteristic-quantity synthesizing means calculates the synthesis 
ratios that are coefficients for adjusting the ratios at which the object 
5 characteristic quantities are synthesized on the basis of the areas of objects 
included in said object characteristic quantities and generates synthesized 
characteristic quantities from said synthesis ratios and said object characteristic 
quantities. 

24. An object-tracking device according to claim 23, wherein said 
characteristic-quantity synthesizing means limits the synthesis ratios within a 
predetermined range on the basis of the variations in the areas of objects. 

25. An object-tracking device according to claim 4, wherein said 
characteristic-quantity synthesizing means receives zone characteristic 
quantities together with object characteristic quantities from the 
characteristic-quantity generating means, calculates synthesized characteristic 

5 quantities within the range of the variations in the areas of objects based on the 
received zone characteristic quantities and object characteristic quantities, and 
provides the synthesized characteristic quantities that have the highest degrees 
of similarity to the zone characteristic quantities of the object zones of interest. 

26. An object-tracking device according to claim 4, wherein said object 
characteristic quantity includes an image template representative of the shape 
and color of an object, and 

said characteristic-quantity synthesizing means decides the 
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5 back-to-belly relation of each of the objects from the image templates and zone 
characteristic quantities and obtains the synthesized characteristic quantities by 
synthesizing the image templates based on the respective decided back-to-belly 
relations of said objects. 

27. An object-tracking device according to claim 4, wherein said 
correspondence-determining means is provided with 

a correspondence-calculating means for calculating the combination of 
objects and object zones which have the highest similarity from all the possible 
5 combinations of the objects and object zones that are possibly associated in 
corresponding relationship based on said synthesized characteristic-quantity 
information, said zone characteristic-quantity information and said first 
zone-correspondence information, selecting the calculated combination of 
objects and object zones as an optimum combination and generating the 

1 0 optimum-correspondence information that indicates the optimum corresponding 
relationship between objects and object zones, and 

a correspondence-deciding means for determining the corresponding 
relationship between objects and object zones on the basis of said first 
zone-correspondence information and said optimum-correspondence 

15 information and providing the correspondence information that has been 

determined that is the information that includes the corresponding relationship 
that has been decided between objects and object zones. 

28. An object-tracking device according to claim 27, wherein said 
correspondence-calculating means calculates a total degree of similarity for 
each of all the possible combinations of objects and object zones, said total 
degree of similarity being a sum of the degrees of similarity between the 
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characteristic quantities of object zones and synthesized characteristic 
quantities within each combination, and decides the combination that has the 
highest similarity based on the combination having the highest, total degree of 
similarity, of said all the possible combinations. 

29. An object-tracking device according to claim 27, wherein said first 
zone-correspondence information includes the information about an 
at-rest/in-motion state that indicates whether an object zone is at rest or in 
motion, and 

said correspondence-calculating means excludes the combination of 
the object and object zone that is indicated as being at rest in said information 
about an at-rest/in-motion state from said all possible combinations. 

30. An object-tracking device according to claim 27, wherein if the degrees 
of combined similarity that can be obtained from the degrees of similarity of the 
sets of the objects and object zones, said sets of the objects and object zones 
making up the combinations decided to be ranked as the highest similarity, are 
equal to or lower than a predetermined threshold, then said 
correspondence-calculating means selects the combinations of the degrees of 
combined similarity within said predetermined threshold, from the combinations 
of the degrees of combined similarity ranked as the highest similarity of all 
possible combinations of objects and object zones, includes the corresponding 
relationship of objects and object zones common to the selected combinations, 
into the optimum-correspondence information as optimum correspondences, 
and further, for the objects and object zones having the corresponding 
relationship that are not included in said corresponding relationship of the object 
and object zone common to said selected combinations, includes the 
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information indicating that there are no optimum correspondence between the 
objects and object zones, into the optimum-correspondence information, 

for the objects not indicated as having no optimum corresponding 
relationship to any object zones in said optimum-correspondence information, 
said correspondence-deciding means provides the information indicating the 
corresponding relationship of objects and object zones included in said 
optimum-correspondence information as the correspondence information that 
has been determined, and 

for the objects indicated as having no optimum corresponding 
relationship to any object zones in said optimum-correspondence information, 
said correspondence-deciding means provides the information indicating the 
corresponding relationship of objects and object zones included in said first 
zone-correspondence information as the correspondence information that has 
been determined. 

31 . An object-tracking device according to claim 28, wherein if the degrees 
of combined similarity that can be obtained from the degrees of similarity of the 
sets of the objects and object zones, said sets of the objects and object zones 
making up the combinations decided to be ranked as the highest similarity, are 
equal to or lower than a predetermined threshold, then said 
correspondence-calculating means selects the combinations of the degrees of 
combined similarity within said predetermined threshold, from the combinations 
of the degrees of combined similarity ranked as the highest similarity of all 
possible combinations of objects and object zones, includes the corresponding 
relationship of objects and object zones common to the selected combinations, 
into the optimum-correspondence information as optimum correspondences, 
and further, for the objects and object zones having the corresponding 
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relationship that are not included in said corresponding relationship of the object 

and object zone common to said selected combinations, includes the 
15 information indicating that there are no optimum correspondence between the 

objects and object zones, into the optimum-correspondence information, 
for the objects not indicated as having no optimum corresponding 

relationship to any object zones in said optimum-correspondence information, 

said correspondence-deciding means provides the information indicating the 
20 corresponding relationship of objects and object zones included in said 

optimum-correspondence information as the correspondence information that 

has been determined, and 

for the objects indicated as having no optimum corresponding 

relationship to any object zones in said optimum-correspondence information, 
25 said correspondence-deciding means provides the information indicating the 

corresponding relationship of objects and object zones included in said first 

zone-correspondence information as the correspondence information that has 

been determined. 

32. An object-tracking device according to claim 27, wherein said state of 
tracking includes a state of parting that is a transient state through which an 
object zone parts into a plurality of object zones, and 

said correspondence-deciding means determines the corresponding 
5 relationship between objects and object zones to be indicated in the 

optimum-correspondence information only for the object zones that exhibit a 
state of parting as their states of tracking. 

33. An object-tracking device according to claim 27, wherein said state of 
tracking includes a state of parting that is a transient state through which an 
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object zone parts into a plurality of object zones, and 

said correspondence-deciding means provides the correspondences 
between objects and object zones included in the first zone-correspondence 
information as the correspondence information that has been determined only 
for the object zones that exhibit states other than the state of parting while in 
their tracking states. 

34. An object-tracking method for tracking an object based on image 
information, comprising steps of: 

synthesizing object characteristic quantities, which represent 
characteristic quantities of respective objects included in said image information, 
to generate a synthesized characteristic quantity, and 

establishing corresponding relationship between object or objects and 
object zone on the basis of the degree of similarity between said synthesized 
characteristic quantity and characteristic quantity of said object zone, wherein 
said object zone is a region extracted from said image information and including 
said object or objects. 

35. An object-tracking method according to claim 34, including steps of 
synthesizing characteristic quantities for each of all required 

combination of a plurality of objects to generate a synthesized characteristic 
quantity, and 

establishing corresponding relationship between object or objects and 
object zone through comparison of the generated synthesized characteristic 
quantity and zone characteristic quantity that represents the characteristic 
quantity of said object zone. 
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36. An object-tracking method according to claim 35, including steps of 
extracting an object zone from said image information and providing the 

object-zone information that includes image information about said object zone, 
deciding a state of tracking representative of a relative position with 

respect to another object for every object or object zone, 

generating zone characteristic quantities, object characteristic quantities 

through the use of said image information, said object-zone information, and the 

decision results, and 

generating synthesized characteristic quantities through the use of said 

object characteristic quantities and said decision results. 

37. An object-tracking method according to claim 36, including steps of 
deciding the state of tracking of every object or every object zone based 

on said object-zone information and the correspondence information that has 
been determined that indicates the corresponding relationship between object 
zones and objects prior to the present and providing the first 
zone-correspondence information indicative of the corresponding relationship 
between the objects and object zones and the states of tracking, 

generating zone characteristic quantities and object characteristic 
quantities based on the present image information, said object-zone information, 
said first zone-correspondence information and said correspondence 
information that has been determined, 

generating a synthesized characteristic quantity that functions as a 
candidate to be placed in corresponding relationship to each object zone on the 
basis of said object characteristic quantities and said first zone-correspondence 
information and providing the synthesized characteristic-quantity information, 
which is the information that includes said synthesized characteristic quantities 
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and the corresponding relationship between synthesized characteristic 
quantities and objects used for generating the synthesized characteristic 
quantities of interest, and 

20 establishing correspondences between objects and object zones based 

on said first zone-correspondence information, zone characteristic-quantity 
information, which is the information indicative of said zone characteristic 
quantities, and said synthesized characteristic-quantity information, and 
providing said correspondence information that has been determined at 

25 present. 

38. An object-tracking method according to claim 36, wherein said state of 
tracking includes at least one of or one of combinations of a stand-alone state, 
in which only a single object is present in an object zone, a crossover state, in 
which a plurality of objects are present in a single object zone, and a state of 

5 parting that is a transient state through which an object zone parts into a 
plurality of object zones. 

39. An object-tracking method according to claim 37, wherein said state of 
tracking includes at least one of, or one of combinations of a stand-alone state, 
in which only a single object is present in an object zone, a crossover state, in 
which a plurality of objects are present in a single object, and a state of parting 

5 that is a transient state through which an object zone parts into a plurality of 
object zones. 

40. An object-tracking method according to claim 37, including steps of 
generating, as a zone characteristic quantity, at least one of, or one of 

the combinations of the color histograms, areas, image templates and color 
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histograms normalized with respect to respective areas, of object zones, and 
seeking the object zones corresponding to said objects from said first 
zone-correspondence information, and generating, as an object characteristic 
quantity, at least one of, or one of the combinations of the color histograms, 
areas, image templates and color histograms normalized with respect to 
respective areas of said object zones. 

41 . An object-tracking method according to claim 37, including steps of 
storing said object-zone information, 

tracking an object on the basis of said object-zone information, the 
correspondence information that has been determined and the object-zone 
information prior to the present and providing a second zone-correspondence 
information indicative of the correspondence between the object and object 
zone, and 

deciding the state of tracking an object on the basis of said second 
zone-correspondence information, said object-zone information and said 
object-zone information prior to the present and providing said first 
zone-correspondence information. 

42. An object-tracking method according to claim 41 , including steps of 
sorting objects and object zones into a plurality of classes by: grouping 

the objects based on those objects which have a common region in the 
corresponding object zone, to enroll said objects and the corresponding object 
zones in one class; and for the objects that correspond to the object zones that 
differ from the object zones corresponding to any other objects, enrolling the 
objects and the corresponding object zone in one class, based on at least one 
of, or one of combinations of the corresponding relationship between objects 
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and object zones, the distances between object zones, and the duration period 
1 0 for on-parting of object zones calculated from said second 

zone-correspondence information and said object-zone information; and 
deciding the state of tracking based on the classified class. 

43. An object-tracking method according to claim 42, wherein said state of 
tracking includes a state of parting that is a transient state through which a 
single object zone parts into a plurality of object zones, and 

said object-tracking method includes steps of 
5 deciding that a classified class meets the condition of being in the state 

of parting if the class includes two or more object zones, and 

if a class meets the condition of being in the state of parting, deciding 
the state of parting based on the states of tracking the objects and object zones 
included in the class of interest. 

10 

44. An object-tracking method according to claim 43, including a step of: 

if the classified class meets the condition of being in the state of parting 
and further meets at least one of, or one combination of the conditions that said 
class includes two or more objects, that the distances between the object zones 
5 included in said class exceed a predetermined threshold, and that the 
continued periods of separation of the object zones included in said class 
exceed a predetermined threshold, deciding the state of parting based on the 
states of tracking the objects and object zones included in said class. 

45. An object-tracking method according to claim 43, wherein said state of 
tracking includes the state of parting and a stand-alone state in which a single 
object is present in an object zone, and said object-tracking method includes a 
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step of 

deciding that the states of tracking of the object and object zone are the 
stand-alone state if the classified class includes a single object and also neither 
of the states of tracking of the object and the object zone included in the class is 
the state of parting. 

46. An object-tracking method according to claim 44, wherein said state of 
tracking includes the state of parting and a stand-alone state in which a single 
object is present in an object zone, and said object-tracking method includes a 
step of 

deciding that the states of tracking of the object and object zone are the 
stand-alone state if the classified class includes a single object and also neither 
of the states of tracking of the object and the object zone included in the class is 
the state of parting. 

47. An object-tracking method according to claim 43, wherein said state of 
tracking includes the state of parting and the crossover state in which a plurality 
of objects correspond to a single object zone, and said object-tracking method 
includes a step of 

deciding that the states of tracking of the objects and object zone 
included in the classified class are the crossover state if the class includes two 
or more objects and neither of the states of tracking of the objects and the 
object zone included in the class is the state of parting. 

48. An object-tracking method according to claim 44, wherein said state of 
tracking includes the state of parting and the crossover state in which a plurality 
of objects correspond to a single object zone, and said object-tracking method 
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includes a step of 

deciding that the states of tracking of the objects and object zone 
included in the classified class are the crossover state if the class includes two 
or more objects and neither of the states of tracking of the objects and the 
object zone included in the class is the state of parting. 

49. An object-tracking method according to claim 37, including steps of 
extracting zone characteristic quantities from said image information, 

said object-zone information and said first zone-correspondence information 
and providing the zone characteristic-quantity information, which is the 
information indicative of said zone characteristic quantities, 

storing said object characteristic quantities and selecting the stored 
object characteristic quantities to be provided as required, and 

updating said stored object characteristic quantities on the basis of said 
zone characteristic-quantity information, said first zone-correspondence 
information or the correspondence information that has been determined and 
the object characteristic quantities generated prior to the present. 

50. An object-tracking method according to claim 49, wherein said state of 
tracking includes the state of parting that is a transient state through which an 
object zone parts into a plurality of object zones, and said object-tracking 
method includes steps of 

including, into the zone characteristic-quantity information, the 
information which indicates that there is no need to establish corresponding 
relationship to any objects, for the object zones that are indicated as having 
states other than the state of parting while in their tracking states, and 

excluding the object zones, in which the zone-correspondence 



information indicates that there is no need of establishing the corresponding 
relationship to any objects, from the establishment of corresponding 
relationship. 

51 . An object-tracking method according to claim 49, wherein said state of 
tracking includes a stand-alone state in which a single object is present in an 
object zone, and said object-tracking method includes steps of 

deciding whether or not the state of tracking is the stand-alone state 
based on said first zone-correspondence information or correspondence 
information that has been determined, and 

skipping an update of the stored object characteristic quantity if the 
state of tracking of an object is any of the states other than the stand-alone 
state. 

52. An object-tracking method according to claim 37, including steps of 
determining all possible combinations of objects and object zones on 

the basis of said object characteristic quantities and said first 
zone-correspondence information, and 

synthesizing object characteristic quantities to generate synthesized 
characteristic quantities only for the determined combinations of objects and 
object zones. 

53. An object-tracking method according to claim 37, including steps of 
finding synthesis ratios, which are the coefficients for adjusting the 

ratios of said object characteristic quantities to be synthesized, and 

generating synthesized characteristic quantities on the basis of said 
synthesis ratios and object characteristic quantities. 
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54. An object-tracking method according to claim 37, including steps of 
receiving zone characteristic quantities together with object 

characteristic quantities, calculating synthesized characteristic quantities for 
arbitrary synthesis ratios based on the received zone characteristic-quantity 
information and the object characteristic quantities, and providing the 
synthesized characteristic quantity corresponding to the highest degree of 
similarity between the zone characteristic quantity and the calculated 
synthesized characteristic quantity. 

55. An object-tracking method according to claim 37, wherein said state of 
tracking includes a state of parting, said state of parting being a transient state 
through which an object zone parts into a plurality of object zones, and said 
object-tracking method includes a step of 

generating synthesized characteristic quantities only for the object 
zones indicated as being in the state of parting while in their tracking states. 

56. An object-tracking method according to claim 37, wherein said object 
characteristic quantity includes an area of an object, and said object-tracking 
method includes steps of 

calculating a synthesis ratio, which is a coefficient for adjusting the 
ratios of the object characteristic quantities to be synthesized, on the basis of 
the areas of objects and generating a synthesized characteristic quantity from 
the calculated synthesis ratio and object characteristic quantities. 

57. An object-tracking method according to claim 56, wherein said 
synthesis ratio is restricted within a range predetermined on the basis of the 
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variations of the object areas. 

58. An object-tracking method according to claim 37, including steps of 
receiving said zone characteristic quantities together with said object 

characteristic quantities, generating synthesized characteristic quantities within 
the range of variations in the object areas based of the received zone 
characteristic quantities and the object characteristic quantities, and providing 
the synthesized characteristic quantity which has the highest degree of 
similarity to the zone characteristic quantity of the object zone of interest. 

59. An object-tracking method according to claim 37, wherein said object 
characteristic quantity includes an image template, which describes a shape 
and/or color of an object, and said object-tracking method includes steps of 

deciding the back-to-belly relations of the objects on the basis of the 
image templates and zone characteristic quantities, and synthesizing the image 
templates based on said decided back-to-belly relations of the objects to obtain 
a synthesized characteristic quantity. 

60. An object-tracking method according to claim 37, including steps of 
calculating, based on said synthesized characteristic-quantity 

information, said zone characteristic-quantity information and said first 
zone-correspondence information, the combination of objects and object zones 
which has the highest degree of similarity from all the possible combinations of 
objects and object zones that can be associated in corresponding relationship, 
selecting the calculated combination of objects and object zones as the objects 
and object zone of optimum correspondence, and generating the 
optimum-correspondence information that indicates the optimum 
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1 0 correspondence relation of the objects and object zone, and 

determining corresponding relationship between objects and object 
zones based on said first zone-correspondence information and said 
optimum-correspondence information and providing the correspondence 
information that has been determined, which is the information inclusive of the 

15 determined corresponding relationship between objects and object zones. 

61 . An object-tracking method according to claim 60, including steps of 
calculating a total degree of similarity for all possible combinations of 

objects and object zones, wherein said total degree of similarity is a sum of the 
degrees of similarity between characteristic quantities of object zones and 
5 synthesized characteristic quantities in each of the combinations, and deciding 
that the combination of the highest total degree of similarity of all said possible 
combinations is the combination of the highest similarity. 

62. An object-tracking method according to claim 60, wherein said first 
zone-correspondence information includes the information about an 
at-rest/in-motion state that indicates whether an object zone is at rest or in 
motion, and said object-tracking method includes a step of excluding, from all 

5 the possible combinations of objects and object zones, the combination of the 
object and object zone indicated as being in the at-rest state by said information 
about an at-rest/in-motion state. 

63. An object-tracking method according to claim 60, including steps of: 
if the degrees of combined similarity, obtained from the degrees of 

similarity of the sets of objects and an object zones that make up the 
combination decided as a combination of the highest similarity, is equal to or 

117 



lower than a predetermined threshold, then selecting the combinations of the 
degrees of combined similarity within a predetermined threshold from the 
combinations of the degrees of combined similarity ranked as the highest 
similarity, of all the possible combinations of objects and object zones, including 
the corresponding relationship between objects and object zones common to 
the selected combinations into the optimum-correspondence information as 
optimum correspondences, and further including the information that indicates 
absence of optimum correspondence between any object and object zone in the 
optimum-correspondence information, for the object and object zone in the 
correspondence relation that is not included in the corresponding relationship of 
the objects and object zones common to said selected combinations; 

for the objects not indicated in said optimum-correspondence 
information that are absent from the optimum corresponding relationship to 
object zones, providing the information indicating the corresponding relationship 
between the objects of interest and object zones included in said 
optimum-correspondence information, as the correspondence information that 
has been determined; and 

for the objects indicated in said optimum-correspondence information 
that are absent from the optimum corresponding relationship to object zones, 
providing the information indicating the corresponding relationship between the 
objects of interest and object zones included in the first zone-correspondence 
information, as the correspondence information that has been determined. 

64. An object-tracking method according to claim 61 , including steps of: 

if the degrees of combined similarity, obtained from the degrees of 
similarity of the sets of objects and object zones that make up the combination 
that is decided as the combination of the highest similarity, is equal to or lower 
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than a predetermined threshold, then selecting the combinations of the degrees 
of combined similarity within a predetermined threshold from the combinations 
of the degrees of combined similarity ranked as having the highest similarity, 
from all the possible combinations of objects and object zones, including the 
corresponding relationship between objects and object zones common to the 
selected combinations in the optimum-correspondence information as optimum 
correspondences, and further including the information that indicates absence 
of an optimum correspondence between any object and object zone in the 
optimum-correspondence information, for the object and object zone in the 
correspondence relation that is not included in the corresponding relationship of 
the objects and object zones common to said selected combinations; 

for the objects not indicated in said optimum-correspondence 
information that are absent from the optimum corresponding relationship to 
object zones, providing information indicating the corresponding relationship 
between the objects of interest and object zones included in said 
optimum-correspondence information, as the correspondence information that 
has been determined; and 

for the objects indicated in said optimum-correspondence information 
that are absent from the optimum corresponding relationship to object zones, 
providing the information indicating the corresponding relationship between the 
objects of interest and object zones included in the first zone-correspondence 
information, as the correspondence information that has been determined. 

65. An object-tracking method according to claim 60, wherein said state of 
tracking includes a state of parting, said state of parting being a transient state 
through which an object zone parts into a plurality of object zones, and said 
object-tracking method includes a step of 
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determining the corresponding relationship between objects and object 
zones to be identical with those indicated in the optimum-correspondence 
information only for the object zones that are indicated as having the state of 
parting while in their tracking states. 

66. An object-tracking method according to claim 60, wherein said state of 
tracking includes a state of parting, said state of parting being a transient state 
through which an object zone parts into a plurality of object zones, and said 
object-tracking method includes a step of 

providing the correspondences between objects and object zones 
included in the first zone-correspondence information as the correspondence 
information that has been determined only for the object zones indicated as 
having the state other than the state of parting while in their tracking states. 

67. An object-tracking program for tracking an object based on image 
information, said program operating a computer to execute processes of 

receiving image information, 

synthesizing object characteristic quantities that represent characteristic 
quantities of respective objects included in said received image information and 
generating a synthesized characteristic quantity, and 

establishing a correspondence between said objects and an object 
zone based on the degree of similarity between a characteristic quantity of said 
object zone and said synthesized characteristic quantity, said object zone being 
a region that is extracted from said image information and also includes said 
objects. 

68. An object-tracking program for establishing correspondences between 
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objects and object zones included in received image information, said program 
operating a computer to execute processes of 
receiving image information, 
5 extracting object zones from said received image information and 

providing the object-zone information inclusive of image information about said 
object zones, 

deciding the state of tracking with each object or object zone on the 
basis of said object-zone information and the correspondence information that 

10 has been determined indicating corresponding relationship of the objects and 
object zones prior to the present and providing first zone-correspondence 
information, which indicates the corresponding relationship of said object zones 
and objects and the states of tracking, 

generating the zone characteristic quantities, which represent the 

1 5 characteristic quantities of the object zones, and the object characteristic 

quantities, which represent the characteristic quantities of the objects through 
the use of said image information, said object-zone information and said first 
zone-correspondence information, 

synthesizing characteristic quantities for all required combinations of a 

20 plurality of objects to generate each of synthesized characteristic quantities 
based on said object characteristic quantities and said first 
zone-correspondence information, and providing synthesized 
characteristic-quantity information, which is the information that includes said 
synthesized characteristic quantities and the corresponding relationship 

25 between the objects used for generating synthesized characteristic quantities 
and the synthesized characteristic quantities, and 

associating said objects and object zones in corresponding relationship 
based on said first zone-correspondence information, said zone 



characteristic-quantity information and said synthesized characteristic-quantity 
30 information, and providing said correspondence information that has been 
determined for the present. 

69. An object-tracking program for establishing correspondences between 
objects and object zones included in received image information, said program 
operating a computer to execute processes of 
receiving image information, 
5 deciding the state of tracking each object or each object zone on the 

basis of said object-zone information and the correspondence information that 
has been determined indicating the corresponding relationship of the objects 
and object zones prior to the present, and providing first zone-correspondence 
information, which indicates the corresponding relationship of said object zones 

1 0 and objects and the states of tracking, 

generating the zone characteristic quantities, which represent the 
characteristic quantities of the object zones, and the object characteristic 
quantities, which represent the characteristic quantities of the objects through 
the use of said image information, said object-zone information and said first 

1 5 zone-correspondence information, and 

while taking each of the objects as a target, deciding the 
correspondences between said objects and object zones through designating 
the correspondences between objects and object zones included in said first 
zone-correspondence information as said correspondences between said 

20 objects and object zones, for the objects that are decided to be in states other 
than a state of parting while in their tracking states, said state of parting being a 
transient state through which an object zone parts into a plurality of object 
zones; for the objects that are decided to be in the state of parting while in their 
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tracking states, synthesizing characteristic quantities for all required 
combinations of a plurality of objects on the basis of said object characteristic 
quantities and said first zone-correspondence information to generate 
respective synthesized characteristic quantities; and comparing each of 
synthesized characteristic quantities with a zone characteristic quantity to 
associate the objects corresponding to the synthesized characteristic quantity, 
which has the highest degree of similarity to an object zone, with the object 
zone of interest to be placed in corresponding relationship. 
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